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1 METHOD AND APPARATUS FOR VALIDATING 

2 CROSS-ARCHITECTURE ISA EMULATION 

3 Technical Field 

4 The technical field is emulation of computer instruction sets. 

5 Background 

6 Cross-architecture emulation is needed when running native applications on a 

7 target platform, or computer, that uses an instruction set architecture (ISA) different from 

8 the ISA for which the native applications were initially intended. The developer of a 

9 cross-architecture emulation product wants to know that the emulator correctly translates 

10 or interprets the native applications. Various schemes exist to verify cross-architecture 

1 1 emulation. However, these schemes cannot easily, if at all, comprehensively test the 

12 emulation process. 

1 3 To improve the emulation process, a binary translation product that runs on the 

1 4 target platform may be used to emulate native instructions running on a native, or legacy, 

15 platform. Binary translation automatically translates binary code from the native 

16 instruction set to binary code for the target platform without the need for high-level 

17 source code. As with any emulation product, a developer of the binary translation 

1 8 product may desire to verify the accuracy of emulation from the native ISA to an ISA 

1 9 operating on a target platform. 

20 One conventional approach to verifying cross-architecture emulation is to send 

21 as many native applications as possible through the binary translation product, or 

22 emulator, and then verify that the outputs produced by the binary translation product are 

23 identical to corresponding outputs produced by the native applications running on the 

24 native platform. This conventional approach has several disadvantages. First, the binary 

25 translation product designer cannot know whether testing is complete because the native 

26 applications are usually compiler-generated and the machine code generated by the binary 

27 translation product only includes a subset of the instruction set architecture. Machine 

28 instructions that are not in this subset will never be generated in the compiler so that 

29 some binary instructions are never tested through the emulation process. Second, running 

30 an application through the binary translation product on the target platform and running 

3 1 the same application through the native platform is cumbersome. A separate process may 

32 then be needed to verify the results. Third, when an error is detected, pinpointing the 
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1 exact machine instruction that caused the error may be difficult or impossible. In 

2 addition, replication of the emulation error may be impossible to achieve. Fourth, 

3 execution of some applications is time- or system-sensitive such that the result of an 

4 execution may not be reproducible, which adds to the difficulty ov verifying binary 

5 emulation. 

6 Summary 

7 A method and an apparatus allows complete and efficient verification of cross- 

8 architecture ISA emulation. A random verification framework runs concurrently on two 

9 different computer architectures. The framework operates without regard to existing 

10 native applications and relies instead on binary instructions in a native ISA. The 

1 1 framework is able to determine emulation errors at a machine instruction level. The 

12 emulation errors are then easily identifiable and reproducible. 

13 A random code generator generates one or more sequences of native machine 

14 instructions and corresponding initial machine states in a pseudo-random fashion. The 

15 native instructions are generated from an entire set of the native ISA. The instructions 

1 6 and the state information are provided to initialize a native computer architecture. The 

1 7 same instructions and state information are provided using standard machine-to-machine 

1 8 languages, such as TCP/IP, for example, to a target computer architecture. The target 

19 computer architecture may be embodied as an actual hardware device, or may be a 

20 simulation of a yet-to-be-built computer architecture. The target computer architecture 

2 1 includes a binary emulator that translates the native instructions into binary instructions 

22 executable on the target computer architecture. The final states of the native and the 

23 target computer architectures are gathered, and a verification engine compares the results. 

24 Any differences may indicate an emulation error or failure. 

25 The random verification framework may be run continuously to test emulation of 

26 the complete set of instructions from the native ISA. Even the least-used machine 

27 instructions are tested by the framework. Further, the framework automates the 

28 emulation verification process. Inter-machine communications allow the native and the 

29 target computer architectures to process the same machine instructions from the same 

30 initial states. Any inconsistencies in the final produced states indicate the emulation 
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1 error. The framework can then easily pinpoint the exact machine instruction, register 

2 number and input machine state that caused the emulation error, thereby significantly 

3 reducing the amount of time required for debugging. Finally, the emulation errors 

4 detected using this framework are easily reproducible. 

5 Description of the Drawings 

6 The detailed description will refer to the following drawings, in which like 

7 numerals refer to like objects, and in which: 

8 Figure 1 is a block diagram of an example of an apparatus for validating cross- 

9 architecture ISA emulation; and 

1 0 Figure 2 is a flowchart illustrating an operation of the apparatus of Figure 1 . 

1 1 Detailed Description 

12 A method and an apparatus allow complete and efficient verification of cross- 

1 3 architecture ISA emulation. A random verification framework runs concurrently on two 

14 different computer architectures. The framework is able to determine emulation errors 

15 at a machine instruction level. The emulation errors are then easily identifiable and 

16 reproducible. 

17 Figure 1 is a block diagram of an embodiment of an apparatus for validating 

1 8 cross-architecture instruction set architecture (ISA) emulation. A framework 1 0 includes 

1 9 a random code generator (RCG) 20 that generates native machine instruction level code 

20 and an initial machine state. The machine instruction level code (i.e. , sequences of binary 

21 instructions) and initial machine state are provided to a native ISA platform 11, which 

22 includes a first, or X, execution engine 30. The X execution engine 30 is capable of 

23 executing the instruction level code without emulation or translation. 

24 The RCG 20 may include a probability file that is used to pseudo-randomly 

25 generate the machine instruction level code. The RCG 20 also provides the machine 

26 instruction level code and the initial machine state to a non-native ISA or target platform 

27 12, which includes a second, or Y, execution engine 40. To execute the machine 

28 instruction level code, the Y execution engine 40 emulates or translates the machine 

29 instruction level code using an emulator 45 . 
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1 In an alternative embodiment, the emulator 45 may operate on a target platform 

2 simulator (not shown), which in turn operates on the native platform 1 1 . 

3 The X execution engine 3 0 and the Y execution engine 40 may execute the same 

4 machine instruction level code concurrently. The execution engines may perform 

5 information transfer using any standard machine-to-machine protocol, such as TCP/IP, 

6 for example. 

7 The X execution engine 30 and the Y execution engine 40 each will provide a 

8 final machine state. The final machine states are sent to a verification engine 50. The 

9 verification engine 50 compares the final machine states to determine any differences. 

1 0 Such differences may indicate an error in emulation of the native machine instruction 

1 1 level code. In particular, the verification engine 50 is able to pinpoint the exact machine 

1 2 instruction, register number and input machine state that caused the emulation error. The 

13 exact source of the emulation error may be particularly easy to locate because the 

1 4 instruction sequence that generated such an emulation error is typically a short sequence 

1 5 of binary instructions. Hence, a review of information saved when an emulation failure 

16 occurs allows the test designer to quickly pinpoint the exact cause of the emulation 

17 failure. 

18 In an example, if a binary instruction to be emulated comprised ADD Rl and R2, 

19 R3, the verification engine 50 would determine if the result written to the register R3 

20 using the emulator 45 and the Y execution engine 40 was the same as that written to the 

2 1 register R3 using the X execution engine 30. As long as the final state produced by the 

22 X execution engine 30 is the same as the final state produced by the Y execution engine 

23 40, the emulation of the binary instruction with a given specific initial machine state was 

24 correct. 

25 The framework 10 relies on pseudo-random generation of machine level 

26 instructions to comprehensively test the correct emulation of native applications on a 

27 second computer architecture. The machine level instructions may be generated 

28 according to many different random generation schemes. In an embodiment, each 

29 machine level instruction is assigned a probability. In particular, a specific instruction 

30 to be generated is controlled by an input probability file, which is a list of pre-defined 
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1 machine instructions, each with an associated probability. The machine instructions are 

2 arranged in a hierarchy form and divided into segments based on a function of the 

3 instruction. A first such segmentation may include floating instructions and CPU 

4 instructions. Continuing, the CPU instructions may be further segmented according to 

5 arithmetic/logic, immediate, memory, system, memory management, and branch 

6 instructions, and other CPU instructions. The arithmetic/logic instructions may be 

7 segmented into ADD, SUB , AND, OR, and XOR instructions, and other arithmetic/logic 

8 instructions, for example. Each hierarchical instruction is assigned a probability such that 

9 a cumulative probability along any hierarchical path equals 1.0. 

10 The instruction sequence generation process is pseudo-random because any 

1 1 random code generator must operate according to a pre-determined algorithm. Further 

1 2 limitations also lead to a loss of true randomness. For example, certain memory regions 

13 may be inaccessible to the random code generator. 

14 An example probability file that controls the pseudo random instruction 

15 generation follows. In the example, expressions such as pr_xxx refer to a probability 

1 6 value for binary instruction xxx. 

1 7 The instructions are arranged in hierarchical segments . 

1 8 pr_cpu 

19 pr_fp 
20 

21 pr_cpu 

22 pr_cpu_arithlog 

23 pr_cpu_immediate 

24 pr_cpu_shexdet 

25 pr_cpu_mem 
26 
27 
28 
29 

30 pr_cpu_arithlog 
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= 0.25 
= 0.00 
= 0.50 
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1 pr_add = 0.04 

2 pr_addl = 0.04 
3 

4 
5 

6 pr_sub = 0.04 

7 pr_subb = 0.04 
8 

9 pr_cpu_immediate =0.25 

10 prjdo =0.15 
11 
12 
13 

14 pr_subi =0.10 

15 

16 

17 

1 8 With the above-assigned probabilities, the RCG 20 will never select a floating 

1 9 point (fp) instruction (i.e., pr_fp = 0.00). Note that the cumulative probability of the first 

20 segment is 1 .00. That is, pr_cpu and pr_fp sum to 1 .00. This cumulative probability rule 

21 is maintained throughout the probability file. 

22 By assigning higher probability values to some instructions and lower probability 

23 values to other instructions, the test designer can ensure that all binary instructions are 

24 eventually tested. 

25 The RCG 20 begins with a seed value and then determines a pseudo random 

26 probability value. For example, a seed value of 1 0 may produce a probability sequence 

27 of 0.34, 0.8, .... Using the cumulative probability in the above example, a random 

28 probability of 0.34 would lead the RCG 20 to generate a cpu_immediate instruction, and 

29 a random probability of 0 . 8 would lead the RCG 20 to generate a pr_cpu_immediate_subi 

30 instruction. 
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1 Figure 2 is a flowchart of a process that may be used by the framework 10 of 

2 Figure 1 . The process includes a main process and sub-processes A and B. Sub-process 

3 A operates on the native platform 1 1 and the X execution engine 30. Sub-process B may 

4 operate in one of at least two ways. First, sub-process B may operate on the binary 

5 emulator 45, which may in turn operate on the target platform 12. Second, sub-process 

6 B may operate on the binary emulator 45, but the binary emulator 45 may operate on a 

7 target platform simulator. The target platform simulator may then operate on the native 

8 platform 11. 

9 The main process begins at 100. In code generate block 110, the RCG 20 

10 generates pseudo-random native machine code and generates a random initial machine 

1 1 state. The native machine code and the initial machine state are provided to the native 

1 2 platform 1 1 , and the native platform 1 1 is initialized using the initial machine state, block 

13 120. The same native machine code and initial machine state are also provided to the 

14 target platform 12, and the target platform 12 is initialized, block 130. The native 

1 5 machine code and initial machine state may be provided to the target platform 1 2 using 

16 TCP/IP protocols, for example. 

17 In block 140, the X execution engine 30 executes the native machine code. 

1 8 Concurrently, the binary emulator 45 emulates the entire Y execution engine 40, which 

19 internally executes the native machine code, block 150. Next, a final machine state SI 

20 is collected in the native platform 1 1 , block 1 60, and a final machine state S2 is collected 

2 1 in the target platform 1 2, block 1 70. 

22 In block 180, the target platform 12 provides the final machine state S2 to the 

23 verification engine 50, using TCP/IP protocols, for example. The verification engine 50 

24 compares the final machine states S 1 and S2. If the final machine states S 1 and S2 are 

25 equal, the process moves to block 220 and ends. If the final machine states SI and S2 are 

26 not equal, the process moves to block 210, and information related to the emulation 

27 failure (i.e., the initial and final states, and the instruction sequence) are written to a file. 

28 The process then moves to block 220 and ends. 

29 The test designer then need only refer to the file to determine the source of the 

30 emulation failure, and to reproduce such emulation failure. 
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1 The terms and descriptions used herein are set forth by way of illustration only 

2 and are not meant as limitations. Those skilled in the art will recognize that many 

3 variations are possible within the spirit and scope of the invention as defined in the 

4 following claims, and there equivalents, in which all terms are to be understood in their 

5 broadest possible sense unless otherwise indicated. 
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1 In The Claims: 

2 1 . An apparatus for pseudo-random testing binary emulation, comprising: 

3 a random code generator that generates an initial machine state and a binary 

4 instruction sequence; 

5 a native architecture execution engine that executes the binary instruction 

6 sequence to produce a final state S 1 ; 

7 a target architecture execution engine that executes the binary instruction 

8 sequence to produce a final state S2, the target architecture execution engine comprising 

9 a binary emulator that emulates the binary instruction sequence according to the target 

10 architecture; and 

11 a verification engine that compares the final state SI and the final state S2, 

1 2 wherein when the final state S 1 and the final state S2 do not match, an emulation failure 

13 has occurred. 

14 2 . The apparatus of claim 1 , wherein the native and the target architecture execution 

15 engines communicate using machine-to-machine communications protocols. 

16 3. The apparatus of claim 2, wherein the communications protocol is a TCP/IP 

17 protocol. 

18 4. The apparatus of claim 1 , wherein when the emulation failure occurs, information 

19 related to the emulation failure is written to a file. 

20 5. The apparatus of claim 1 , wherein the target platform is a simulator. 

21 6. The apparatus of claim 1 , wherein the target platform is a hardware embodiment. 

22 7. The apparatus of claim 1, wherein the random code generator comprises a 

23 probability file comprising probability values for each instruction in a native instructions 

24 set architecture (ISA). 

25 8 . The apparatus of claim 7, wherein the probability values in the probability file are 

26 user-generated. 

27 9. A method for pseudo-random testing of binary emulation, comprising: 

28 generating a pseudo-random binary instruction sequence; 

29 generating an initial machine state; 

30 initializing a native machine according to the initial machine state; 
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1 executing the binary instruction sequence on the native machine to produce a final 

2 state SI; 

3 initializing a target machine according to the initial machine state; 

4 emulating the binary instruction sequence on the target machine; 

5 executing the emulated binary instruction sequence to produce a final state S2; 

6 and 

7 comparing the final state S 1 to the final state S2 to determine an emulation error. 

8 10. The method of claim 9, further comprising: 

9 communicating the initial machine state and the binary instruction sequence to 

1 0 the target machine using a machine-to-machine communication protocol; and 

1 1 communicating the final state S2 to the native machine using the machine-to- 

1 2 machine communication protocol . 

13 11. The method of claim 10, wherein the machine-to-machine communication 

1 4 protocol is a TCP/IP protocol. 

15 12. The method of claim 9, further comprising storing information related to the 

16 emulation failure, the information comprising the initial machine state, the binary 

17 instruction sequence and the final states SI and S2. 

18 13. The method of claim 9, wherein the binary emulation is executed on a simulator. 

19 14. The method of claim 9, wherein the binary emulation is executed on a hardware 

20 device. 

21 15. The method of claim 9, further comprising: 

22 assigning a probability value to each binary instruction in a native instruction set 

23 architecture (ISA); 

24 pseudo-randomly generating a probability value; and 

25 selecting the binary instruction sequence based on the probability value. 

26 16. A method for verifying cross-architecture emulation, comprising: 

27 randomly generating a native instruction sequence and an initial machine state; 

28 providing the native instruction sequence and the initial machine state to a first 

29 platform having a first instruction set architecture (ISA); 
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1 providing the native instruction sequence to a second platform having a second 

2 ISA; 

3 emulating the native instruction sequence according to the second ISA; 

4 executing the native instruction sequence in the first platform, the execution 

5 providing a final state S 1 ; 

6 executing the emulated native instruction sequence on the second platform, the 

7 execution providing a final state S2; and 

8 comparing the final states SI and S2 to determine an emulation failure. 

9 17. The method of claim 16, wherein the native instruction sequence is randomly 

1 0 generated from a set of all instructions in the first ISA. 

11 18. The method of claim 16, wherein the random generation is based on a user- 

1 2 defined value assigned to each instruction in the first ISA. 

13 19. The method of claim 1 6, wherein the emulation of the native instruction sequence 

1 4 and the execution of the emulated native instruction sequence is completed on a binary 

1 5 instruction emulator operating on a simulator which operates on the first platform. 

16 20. The method of claim 16, wherein the emulation is completed on a binary 

17 instruction emulator operating on the second platform. 
18 
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1 Abstract 

2 A method and an apparatus allows complete and efficient verification of cross- 

3 architecture I S A emulation. A random verification framework runs concurrently on two 

4 different computer architectures. The framework operates without regard to existing 

5 native applications and relies instead on binary instructions in a native ISA. The 

6 framework determines emulation errors at a machine instruction level. A random code 

7 generator generates one or more sequences of native machine instructions and 

8 corresponding initial machine states in a pseudo-random fashion. The native instructions 

9 are generated from an entire set of the native ISA. The instructions and the state 

10 information are provided to initialize a native computer architecture. The same 

11 instructions and state information are provided using standard machine-to-machine 

12 languages, such as TCP/IP, for example, to a target computer architecture. A binary 

1 3 emulator then translates the native instructions so that the instructions may be executed 

14 on the target computer. Alternatively, the binary emulator may be embodied as a 

1 5 software routine operating on a simulator, which in turn operates on the native computer 

16 architecture. The final states of the native and the target computer architectures are 

17 gathered, and a verification engine compares the results. Any differences may indicate 

18 an emulation error or failure. The random verification framework may be run 

1 9 continuously to test emulation of the complete set of instructions from the native ISA. 
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